Compensating for Non-Uniform Illumination of Object Fields Captured by a Camera

ABSTRACT

Techniques for modifying data of an image that can be implemented in a digital camera, video image capturing device and other optical systems are provided to correct for non-uniform illumination appearing in data obtained using one or more illumination sources from a two-dimensional photo-sensor. In order to correct for these variations, a small amount of modification data is stored in a small memory within the optical system. According to a specific embodiment, non-uniform illumination correction factors are derived during a calibration procedure by illuminating a surface having uniform optical properties with the non-uniform illumination source and imaging that illuminated surface onto the photo-sensor.

CROSS-REFERENCE TO RELATED APPLICATION

This is related to another application of Shimon Pertsel entitled“Techniques for Modifying Image Field Data Obtained Using IlluminationSources,” being filed concurrently herewith.

FIELD OF THE INVENTION

This invention relates generally to techniques of processing captureddigital imaging data obtained using one or more illumination sources,and, more specifically, to processing binary digital image data obtainedusing one or more illumination sources to correct for variations acrossan imaged optical field such as, for example, to compensate fornon-uniform illumination.

BACKGROUND

Digital cameras image scenes onto a two-dimensional sensor such as acharge-coupled-device (CCD), a complementary metal-on-silicon (CMOS)device or other type of light sensor. These devices include a largenumber of photo-detectors (typically three, four, five or more million)arranged across a small two dimensional surface that individuallygenerate a signal proportional to the intensity of light or otheroptical radiation (including infrared and ultra-violet regions of thespectrum adjacent the visible light wavelengths) striking the element.These elements, forming pixels of an image, are typically scanned in araster pattern to typically generate a serial stream of data of theintensity of radiation striking one sensor element after another as theyare scanned. Color data are most commonly obtained by usingphoto-detectors that are sensitive to each of distinct color components(such as red, green and blue), alternately distributed across thesensor. Non-uniform illumination, and potentially other factors, causesan uneven distribution of light across the photo-sensor, and thus imagedata signals from the sensor include data of the undesired intensityvariation superimposed thereon.

SUMMARY OF THE INVENTION

One or more illumination sources may be used to illuminate an imagefield. An illumination source may, as an example, be a flashillumination device. An illumination source will often be part of theimaging device but may also be a separate device. An illumination sourcemay produce non-uniform illumination across an image field. Non-uniformillumination may be attributed to imperfections in or othercharacteristics of an illumination source, improper alignment of anillumination source in relation to the x-y position of the image planeof the photo-sensor employed, and possibly other factors that may bepresent in a particular system.

The invention offers techniques for modifying image field data tocompensate for non-uniformities in the illumination so as to minimizedegradation of the final adjusted image by these non-uniformities in oneor more illumination sources. The amount of compensation applied to thesignal from each photo-detector element is dependent upon the positionof the element in relationship to the pattern of non-uniformillumination of the image field across the surface of the imagephoto-sensor.

Such non-uniform illumination compensation techniques have applicationto digital cameras and other types of digital image capturing devicesemploying one or more illumination sources but are not limited to suchoptical photo system applications. The techniques may be implemented ata low cost, require a minimum amount of memory, and operate at the samerate as the digital image data being modified is obtained from thephoto-sensor, thereby not adversely affecting the performance of thedigital image processing path. This is accomplished by applyingcorrection factors in real time to the output signals of thephoto-sensor in order to compensate for an undesired intensity variationacross the photo-sensor.

In a specific embodiment, the camera or other optical system iscalibrated by imaging a scene of uniform intensity onto thephoto-sensor, capturing data of a resulting intensity variation acrossthe photo-sensor. The pixel array is logically divided into a grid ofblocks and then average rates of change of the intensity across eachblock are computed. The calibration data needed to correct for theintensity variation is computed as the inverse of the intensityvariation. A reduced amount of data of the undesired non-uniformillumination pattern (or the inverse, the non-uniform illuminationcorrection factors) may be stored in one or more sparse two-dimensionallookup tables. A separate lookup table can be used for each color.

Additional objects, advantages and features of the present invention areincluded in the following description of exemplary embodiments thereof,which description should be taken in conjunction with the accompanyingdrawings. Each patent, patent application, article or publicationreferenced herein is hereby incorporated herein in its entirely for allpurposes by such reference.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 schematically illustrates an digital camera in which thetechniques of the present invention may be utilized;

FIG. 2 is a block diagram of a portion of the electronic processingsystem of the device of FIG. 1;

FIG. 3 schematically illustrates the calibration phase of a specificembodiment of the invention, employing a surface with uniform opticalproperties, a camera with an artificial illumination source, and showingthe path of incident light emitted by the illumination source andreflected light reflected from the surface back to the camera lens;

FIG. 4 is a block diagram setting forth steps in calibration of a cameraor other optical system of interest using an illuminated surface withuniform optical properties;

FIG. 5 is a block diagram setting forth steps in applying storedillumination calibration data to obtain illumination-corrected output;

FIGS. 6A, 6B, 6C, 6D, 6E graphically illustrate the application ofstored illumination calibration data to obtain illumination-correctedoutput;

FIG. 7 schematically illustrates an image capture and modification phasein which an object of interest is imaged, employing the camera with anartificial illumination source, and showing the path of incident lightemitted by the illumination source and reflected light reflected fromthe object of interest back to the camera lens; and

FIG. 8 graphically illustrates the selection of a relevant portion ofthe calibration information.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

When one or more illumination sources are used to illuminate an imagefield, non-uniform illumination across the image field may result in avariation of energy across each pixel of that light pattern. Theseenergy variations are not related to the captured image or other picturedata itself. The variation of illumination across the scene, assumingthe objects in the scene are approximately the same distance from thesource of the flash illumination, has fixed properties. These propertiesare directly related to the physical, optical and electroniccharacteristics of the illuminating flash source. In order to compensatefor this variation in energy across the photo-sensor, each pixel valuecould be combined, such as by multiplication, with a non-uniformillumination correction factor. This factor is unique to each pixel inthe image sensor according to the pixel's geographic location in theimage sensor matrix. In the ideal case, a table of factors could becreated during a calibration procedure that stores the requiredcompensation factor for each pixel of the image in memory. This wouldallow the needed non-uniform illumination compensation to be effected byexecuting one of the following equations with a processing unit in theimage capturing device:PixelOut=PixelIn+F(X, Y)  (1a)orPixelOut=PixelIn*F′(X, Y)  (1b)where,

-   PixelOut=The intensity output of the non-uniform illumination    compensation module; in other words, the corrected pixel;-   PixelIn=The intensity input to the non-uniform illumination    compensation module; in other words, the pixel before correction;-   F(X,Y)=An additive correction factor, having units of intensity,    which depends on the pixel's position expressed in terms of X and Y    rectangular coordinates; and-   F′(X,Y)=A dimensionless multiplicative correction factor, which also    depends on the pixel's position expressed in terms of X and Y    rectangular coordinates.

To calculate the correction factors for the entire image, one of thefollowing equations is executed:CT[x,y]=T[x,y]+IC[x,y],  (2a)orCT[x,y]=T[x,y]*IC′[x,y],  (2b)where CT[x,y] is the illumination-corrected image data set of interestas a function of the position (x,y) of an image data point of interest,T[x,y] is the un-corrected image data set of interest as a function ofthe position (x,y) of an image data point of interest, and IC[x,y] is anadditive illumination correction factor of equation (2a) as a functionof the position (x,y) of a image data point of interest. IC′[x,y] is adimensionless multiplicative illumination correction factor as afunction of the position (x,y) of a image data point of interest, in thealternative equation (2b). Generally speaking, equations (2a) and (2b)represent the image-wide equivalent of equations (1a) and (1b),respectively, which are applied on a pixel by pixel (or pixel block bypixel block) basis. When all of the corrective factors IC[x,y] orIC′[x,y] for a particular image, depending upon which of equations (2a)or (2b) is being used, are listed according to their x,y coordinates,this list represents a two-dimensional mask. The values of that mask atpositions x,y across the image are then combined with the image data atthe same positions x,y across the image.

It would be very costly to implement the process defined by thisequation on an integrated circuit with the storage of correction factorsfor each pixel of the photo-sensor. A large memory would be required tostore a correction factor for each pixel and thus utilize a large areaof silicon for the memory. Multiplication of the pixel values by theindividual stored correction factors can further require a significantamount of silicon area for dedicated circuits to carry out themultiplication and/or can slow down the speed with which corrected dataare obtained. Therefore, the techniques described herein providealternative methods that require very little memory and processing powerbut yet eliminate undesired light patterns from the image that arecaused by artificially illuminating the image scene.

Optical Device Example

An implementation of the techniques of the present invention isdescribed in a digital camera or other digital image acquisition device,where digital data of the image(s) or other captured light pattern(s)obtained using one or more illumination sources are modified on the flyto compensate for intensity variations superimposed across the image dueto non-uniform illumination. In FIG. 1, such a digital camera isschematically shown to include a case 11, an imaging optical system 13,user controls 15 that generate control signals 17, a video input-outputreceptacle 19 with internal electrical connections 21, and a card slot23, with internal electrical connections 25, into which a non-volatilememory card 27 is removably inserted. Data of images captured by thecamera may be stored on the memory card 27 or on an internalnon-volatile memory (not shown). Image data may also be outputted to avideo device, such as a television monitor, through the receptacle 19.The memory card 27 can be a commercially available semiconductor flashelectrically erasable and programmable read-only-memory (EEPROM), smallremovable rotating magnetic disk or other non-volatile memory to whichdigital image data can be stored by the camera. Alternatively,particularly when the camera is taking motion image sequences at thirtyimage frames per second or the like, larger capacity storage media canbe used instead, such as magnetic tape or a writable optical disk.

The optical system 13 can be a single lens, as shown, but will normallybe a set of lenses. An image 29 of a scene 31 is formed as visibleoptical radiation through a shutter 33 onto a two-dimensional surface ofan image sensor 35. An electrical output 37 of the sensor carries ananalog signal resulting from scanning individual photo-detectors of thesurface of the sensor 35 onto which the image 29 is projected. Thesensor 35 typically contains a large number of individualphoto-detectors arranged in a two-dimensional array of rows and columnsto detect individual pixels of the image 29. Signals proportional to theintensity of light striking the individual photo-detectors are obtainedin the output 37 in time sequence, typically by scanning them in araster pattern, where the rows of photo-detectors are scanned one at atime from left to right, beginning at the top row, to generate a frameof digital image data from which the image 29 may be reconstructed. Theanalog signal 37 is applied to an analog-to-digital converter circuitchip 39 that generates digital data in circuits 41 of the image 29.Typically, the signal in circuits 41 is a sequence of individual blocksof digital data representing the intensity of light striking theindividual photo-detectors of the sensor 35.

Processing of the video data in circuits 41 and control of the cameraoperation are provided, in this embodiment, by a single integratedcircuit chip 43. In addition to being connected with the circuits 17,21, 25 and 41, the circuit chip 43 is connected to control and statuslines 45. The lines 45 are, in turn, connected with the shutter 33,sensor 29, analog-to-digital converter 39 and other components of thecamera to provide synchronous operation of them. A separate volatilerandom-access memory circuit chip 47 is also connected to the processorchip 43 for temporary data storage. Also, a separate non-volatilere-programmable memory chip 49 is connected to the processor chip 43 forstorage of the processor program, calibration data and the like. A usualclock circuit 51 is provided within the camera for providing clocksignals to the circuit chips and other components. Rather than aseparate component, the clock circuit for the system may alternativelybe included on the processor chip 43. An illumination source 53 isconnected to, and operates in response to instructions from, theprocessor chip 43.

Sensor 35 may have its large number of pixels logically divided intorectangles of a grid pattern. One way to determine the correction factorfor individual pixels, without having to store such factors for allpixels of the array, is to store them for a representative few of thepixels in each block and then calculate the correction for otherindividual pixels by interpolation, linear or otherwise. That is, thesize of the blocks of the grid pattern are made small enough such thatthe intensity variation of the non-uniform illumination pattern acrossan individual block may be predicted from a few stored values in theblock. For each pixel location, the correction factor is extrapolatedfrom this stored subset. The correction factor extrapolation formula isimplemented as a two dimensional extrapolation responsive to thegeometric distance between the pixel of interest at a current location,and neigboring pixels that are represented by a non-uniform illuminationcorrection factor stored in a limited table of correction factors.

A functional block diagram of the processor chip 43 is shown in FIG. 2.A digital signal processor (DSP) 55 is a key component, controlling boththe operation of the chip 43 and other components of the camera. Butsince the DSP 55 does not extensively process video data, as discussedbelow, it may be a relatively simple and inexpensive processor. A memorymanagement unit 57 interfaces the DSP 55 to the external memory chips 47and 49, and to output interface circuits 59 that are connected to theinput-output connector 19 and to the card slot 23 (FIG. 1) throughrespective circuits 21 and 25.

The flow of digital image data through the block diagram of FIG. 2 fromthe analog-to-digital converter 39 (FIG. 1) is now generally described.The input data in lines 41 is pre-processed in a block 61 and thenprovided as one input to a multiplier circuit 63. Another input 65 tothe multiplier 63 carries data that modifies the incoming video data,the modified video data appearing at an output 67 of the multiplier 63.In this example, the intensity correction data in lines 65 correct forthe effects of lens shading and intensity variations imparted across theimage by camera elements. After further image processing 69, asappropriate, the digital image data are directed through the memorymanagement unit 57 to the output interface circuits 59 and then througheither lines 21 to the input-output receptacle 19 or through lines 25 tothe memory card slot 23 (FIG. 1), or both, of the camera for displayand/or storage.

The intensity correction data in lines 65 are generated by a block ofdedicated processing circuits 71. The block 71 includes circuits 73 thatprovide the (X, Y) position of each image pixel from which video dataare currently being acquired. This pixel position is then used by anintensity correction data calculation circuit 75 to generate themodification factor applied to the multiplier 63. A memory 77 stores alook-up table. In order to reduce the size of the memory 77, only asmall amount of correction data are stored in the look-up table and thecircuits 75 calculate the correction values of individual pixels fromsuch data.

A set of registers 79 stores parameters and intermediate results thatare used by both of the calculation circuits 73 and 75. The calculationcircuits 73 and 75 operate independently of the DSP 55. The DSP couldpossibly be used to make these calculations instead but this wouldrequire an extremely fast processor, if sufficient speed were evenavailable, would be expensive and would take considerable more space onthe chip 43. The circuits 73 and 75, dedicated to performing therequired repetitive calculations without participation by the DSP 55,are quite straightforward in structure, take little space on the chip 43and frees up the DSP 55 to perform other functions. The memory ormemories 77 and 79 storing the image modification data and parametersare preferably a volatile random-access type for access speed andprocess compatibility with other processor circuits so that they can allbe included on a single cost effective chip.

A typical digital imaging system processes data for each of multipledistinct color components of the image. A typical commercial sensoralternates photo-detectors along the rows that are covered with red,green and blue filters. There are several different arrangements of thecolor sensitive photo-detectors that are commercially used. In one sucharrangement, one row contains alternating red and green sensitivephoto-detectors, while the next row contains alternating blue and greensensitive photo-detectors, the photo-detectors also being positionedalong the rows to provide alternating color sensitivity in columns.Other standard arrangements use other combinations of the alternatingcolors.

If there is only one set of correction data for all of the discretecolors being detected, an image modification factor is generated foreach image pixel from that set of data, regardless of the color. This isquite adequate in cases where the variation across the image that isbeing removed by the signal modification affects all colors to the sameor nearly the same degree. However, where the variation is significantlycolor dependent, separate correction factors are preferably used foreach color component.

One desirable flash strobe module is an insulated gate bipolartransistor (IGBT) type, allowing for the intensity of the illuminationlevel to be controlled. A flash strobe module employing asilicon-controlled rectifier (SCR) does not permit effective control offlash intensity.

SPECIFIC EMBODIMENT

In this embodiment, non-uniform illumination correction factors for anoptical photo system of a digital camera, digital video capturing deviceor other type of digital imaging device, are derived during acalibration procedure. This calibration is performed by imaging asurface having uniform optical properties onto the photo-sensor employedby the device being calibrated. One example of such a surface is auniform mid-level gray target. The individual pixel intensity values ofan image of such a target are captured and the slope values for theindividual rectangles of the grid across the photo-sensor are calculatedand stored in a memory within the device being calibrated. Imagemodification data and parameters are generated once for each camera at afinal stage of its manufacture and then are permanently stored in thenon-volatile memory 49 (FIG. 2). These data are then loaded throughlines 81 into the memories 77 and 79 each time the system isinitialized, under control of the DSP 55 operating through control andstatus lines 83.

FIG. 3 schematically illustrates the calibration phase of the operationof the invention according to the this embodiment. An imaging device(e.g., a camera) employing an illumination source (e.g., a flash) iscalibrated by use of an illumination balance reference, which is auniformly reflecting target image (such as a gray card), in order tocorrect for non-uniform illumination by the illumination source. Acamera (or other imaging device) 11 comprising a lens 13 and anillumination source 53 in the form of a flash is used to capture animage of surface 91 with uniform color, absorption/reflection,dispersion and other optical properties thereacross. Light rays 93 aregenerated by the flash 53 and emanate toward the surface 91. Incidentlight rays 93 strike the surface 91, which reflects rays 95. Thereflected rays 95 are imaged by a lens 13 onto the photo-detector withinthe camera 11. The camera 11 then processes the information and uses itto calibrate the non-uniform illumination correction data that willcompensate for non-uniform illumination across an image scene that isproduced by the flash 53. It should be noted that the focal length 97 inthis embodiment is the indicated as a perpendicular distance from thelens 13 to the surface 91.

FIG. 4 is a block diagram setting forth steps of factory calibration. Instep 400, a surface with uniform optical properties is illuminated. Instep 410, image data from the image field of the illuminated uniformsurface is captured. In step 420, illumination correction data aregenerated as a function of position in the image field. In step 430, theillumination correction data are stored in a non-volatile memory of theimaging device.

In two prior patent applications, Publication Number 2004-0032952 A1,filed Aug. 16, 2002, and Publication Number 2004-0257454, filed Dec. 30,2003, intensity variations across the image are compensated bycharacterizing those variations as one or more geometric shapes, such ascircles, ellipses or hyperbolas, and then storing a resulting smallamount of data necessary to characterize the geometric shape or shapes.The correction factor for each pixel may be computed as aone-dimensional function along the geometric distance to a reference onthe image geometric plane. In order to greatly simplify the circuitsthat perform the calculations, the algorithm executed by the circuits 73(FIG. 2) preferably relies upon arithmetic addition, which is a fast andsimple way of computing a correction factor for each pixel based on itsposition.

However, there are situations where it is too difficult or not practicalto represent a non-uniform illumination pattern with desired accuracy byone or a few simple geometric shapes. As an alternative, according to athird application, Publication Number 2005-0041806 A1, filed Feb. 2,2004, the matrix of pixels of a photo-sensor can be logically dividedinto a grid of a large number of contiguous rectangular blocks that eachcontains a fixed number of pixels on a side. During calibration, data ofthe non-uniform illumination pattern on the individual blocks arecalculated and stored, from which a stored data correction factor iscalculated for the individual pixels or blocks of pixels as picture dataare scanned from the photo-sensor, in real time, typically in a rasterscanning pattern.

The calibration data may in some applications be captured and storedwith a resolution that is less than that with which data of an imagefield are normally captured. When capturing data of the uniform screen91 (FIG. 3), for example, data of individual pixels may be combined toprovide one data point per color component for a block of pixels. Thisreduces the amount of calibration data that need to be stored in thecamera and increases the speed with which correction of full resolutionimage data may be made.

FIG. 5 is a block diagram setting forth steps in applying storedillumination correction calibration data to obtainillumination-corrected output. In step 500, stored illuminationcorrection data is retrieved. In step 510, image data of the illuminatedimage field is captured. In step 520, the captured image data is stored.In step 530, the image modification parameters are combined with theimage data to obtain improved, illumination-corrected output.

FIGS. 6A-6E graphically illustrate the application of storedillumination correction calibration data to obtainillumination-corrected output. Each figure provides a plot of intensityagainst position (in either the x or y direction) along a line across aphoto-sensor through its optical center. FIG. 6A depicts the intensityrelative to position of light reflected by a surface with uniformoptical properties. FIG. 6B depicts illumination correction data 98,which is designed to compensate for the non-uniformities seen in FIG.6A. The arithmetic sum of intensity on the photo-sensor and theillumination correction data produces a flat line of constant intensity,as seen in FIG. 6C. In FIG. 6D, an example is provided of possible datafor an image of interest. FIG. 6E provides the illumination-correcteddata for this same intensity pattern.

FIG. 7 schematically illustrates an example of the capture of the imageof interest of step 510. The camera 11, with its lens 13 andillumination source 53, is used to capture data of an object scene ofinterest 94. The object of interest 94 has a length 96 and typicallysubtends a segment 99 of the calibration non-uniform illuminationcorrection data 98. Incident light rays 93 are generated by the flash 53and emanate toward the object 94. Incident light rays 93 strike theobject 94, which reflects reflected rays 95. Reflected rays 95 areimaged by the lens 13 onto the photo-detector. The camera 11 thencombines the image of interest with the calibration data 98 (step 530)to produce an illumination-corrected image of the object of interest 94.Focal length 97 is the indicated distance from lens 13 to object 94. Inthis example, only segment 99 of the calibration data 98 is used toproduce the corrected image. Calibration data 98 not contained withinsegment 99 have no effect on the corrected image generated of object ofinterest 94. A higher zoom setting will result in a smaller portion ofthe image of interest being captured by the imaging device. The anglesubtended by the captured image of interest, i.e., the zoom settingused, is incorporated into the calibration process. The calibrationprocess also incorporates the distance of the imaging device from theimage of interest, which bears an inverse square relationship to theintensity. It is assumed that the non-uniform illumination pattern isindependent of distance from the imaging device, while varying inintensity. Either the correction process is performed pixel by pixel orelse it is performed pixel block by pixel block.

FIG. 8 represents the calibration curve of FIG. 6B, with segment 99indicated as the portion of the calibration curve lying between thedashed lines. Only segment 99 of the calibration data is used incorrecting the image of interest.

Calibration correction information can be executed within the imagingdevice, thus permanently modifying the original, uncorrected image data.Alternatively, the calibration correction information can be storedwithin the imaging device as auxiliary data to be used inpost-processing at a digital image processing service center, or by theuser as part of image enhancement. The corrected image may be previewedon the imaging device's preview screen before it is permanently appliedto the image data. The calibration can be carried out using lowresolution images. Low resolution images will typically suffice forobtaining calibration correction information for a featureless object.

According to the specific embodiment being described herein, each cameraor other optical system is calibrated by imaging a scene of uniformintensity onto the photo-sensor, capturing data of each pixel of aresulting intensity variation across the photo-sensor, logicallydividing the pixel array into a grid of blocks and then calculatingaverage rates of change of the intensity across each block. Theserelatively few intensity slope values, the characteristics of the pixelgrid and the absolute intensity of the first pixel of each scannedframe, characterize the non-uniform illumination intensity variationacross the photo-sensor with a reduced amount of data. It is usuallydesirable to acquire three sets of such data, one set for each primarycolor that is utilized in the picture processing.

The created slope tables and the basic gain is stored in a digitalcamera's nonvolatile memory 49 of FIGS. 1 and 2, for example, during themanufacturing process, and subsequently used to compensate fornon-uniform illumination non-uniformities as previously described.

In cases in which more than one illumination source is in use,correction for non-uniform illumination can be achieved by using theprinciple of superposition. The composite non-uniform illuminationpattern to be corrected is composed of several non-uniform illuminationpatterns superimposed on one another. These patterns are preferablyseparated at calibration time and multiple non-uniform illuminationpatterns are visualized, each with its own center of gravity. (Thecenter of gravity is also known as the optical center or the anchorpoint.) These centers of gravity can then be combined into an “effectivecenter of gravity” and used to form lookup table 77 of FIG. 2, or eachused individually to derive separate look up tables which aresubsequently combined to form lookup table 77. In this latter case, thealgorithm employed to combine these shading correction factors for usein table 77 can be either linear, piece-wise linear, or non-linear.Thus, a large degree of flexibility in choosing non-uniform illuminationcorrection factors with respect to a particular image element locationis provided.

The optical center of a pattern of illumination will not necessarily bealigned with or bear any particular relationship with the opticalgeometry of the imaging device. The pattern of illumination may, forinstance, be incident from one side of the image of interest. Reflectordevices can be employed to attempt to assist but typically cannotprecisely resolve such considerations. Illumination correction patternsare a means for correcting for such issues. Effects of a varying focallength may also be taken into account.

Since the complete optical photo system of the digital imaging device isemployed during the calibration procedure, the correction data alsoinclude correction for any intensity variations across the image thatare caused by lens shading, effects of the optical cavity, the imagesensor and/or its interaction with the incident image light, and thelike, in addition to providing correction for non-uniformities due tonon-uniform illumination by the illumination source. It may be desirableto have separate correction data for the non-uniform illumination of anobject scene. If so, correction data are separately captured for lensshading and the like by imaging the same screen used in acquiringillumination correction data with a non-uniform light source but thistime with uniform illumination across it, such as by one of the methodsdescribed in the previously identified U.S. patent applicationpublication numbers 2004-0032952, 2004-0257454 and 2005-0041806. Whenthe resulting lens shading data are subtracted from the combinedcorrection data obtained with the non-uniform light source according tothis embodiment, on a pixel-by-pixel or pixel-block by pixel-blockbasis, correction data for the non-uniform light source are obtainedwithout components of lens shading and the like.

Conclusion

The present invention provides unique illumination compensation ofdigital images captured from a non-uniformly lit scene. A commoninstance where such compensation is beneficial is the capture of a sceneilluminated by a digital camera's small, built-in, electronic flashunit.

Although the present invention has been described with respect tocertain embodiments, it will be understood that the invention isentitled to protection within the fall scope of the appended claims.

1. A method of calibrating an imaging device that uses an illuminationsource, comprising: illuminating with the illumination source a surfacehaving uniform optical properties thereacross; capturing image data froman image field of the uniform surface as illuminated by the illuminationsource; generating, from the captured image data, illuminationcorrection data as a function of position in the image field; storingthe illumination correction data in the imaging device.
 2. The method ofclaim 1, wherein generating the illumination correction data includesmaintaining a plurality of sets of illumination correction data for theindividual pixels being scanned, one set of data for each of a pluralityof color components of the optical field.
 3. The method of claim 1,wherein generating the illumination correction data is done as afunction of distance from an optical center.
 4. The method of claim 3,wherein the illumination correction data are generated as a function ofdistance from an optical center of an optical system utilized to capturethe image data.
 5. The method of claim 3, wherein the illuminationcorrection data are generated as a function of distance from an opticalcenter of the illumination source.
 6. The method of claim 1, wherein thecapturing, generating, and storing functions are all accomplished on asingle integrated circuit chip.
 7. The method of claim 1, wherein thecapturing, generating, and storing functions are all performed byelectronic circuits dedicated to carrying out these functions.
 8. Themethod of claim 1, wherein the illumination source is included withinthe imaging device.
 9. The method of claim 1, wherein the illuminationsource includes a flash.
 10. The method of claim 9, wherein the flash isincluded within the imaging device.
 11. The method of claim 1, whereinthe illumination correction data are stored in a nonvolatile memorydevice.
 12. The method of claim 11, wherein the nonvolatile memorydevice includes a flash memory device.
 13. The method of claim 1,wherein the imaging device is a still camera.
 14. The method of claim 1,wherein the imaging device is a video camera.
 15. The method of claim 1,wherein the imaging device is a personal digital assistant (PDA). 16.The method of claim 1, wherein the imaging device is a cellulartelephone.
 17. The method of claim 1, wherein the imaging device ishandheld.
 18. A method of modifying electronically captured image fielddata using an imaging device that has been calibrated according to themethod of claim 1, comprising: retrieving the stored illuminationcorrection data; capturing image data of an object image fieldilluminated by the illumination source; and combining the retrievedillumination correction data with the object image data, to obtain anillumination-corrected object image data set, thereby correcting forvariations in the intensity attributable to the illumination sourceplaced across the object image field.
 19. A method of modifyingelectronically captured image field data using an imaging device thathas been calibrated according to the method of claim 1, comprising:retrieving the stored illumination correction data; capturing image dataof an object image field illuminated by the illumination source; andcombining the retrieved illumination correction data with the objectimage data, to obtain an illumination-corrected object image data set,thereby correcting for variations in the intensity attributable to theillumination source placed across the object image field, wherein theresolution of the image data set is greater than the resolution of theillumination correction data.
 20. A method of modifying an image fieldbeing electronically captured, comprising: retrieving storedillumination correction data configured to compensate for variations inintensity across an image field being electronically captured that areattributable to a source of illumination across the image field;capturing image data of the illuminated image field; and combining thestored illumination correction data with the image data, in order toobtain an illumination-corrected image data set, thereby correcting forvariations in intensity across the image field that are attributable tothe illumination source.
 21. The method of claim 20, wherein combiningthe stored illumination correction data with the image data includescombining a portion of the stored illumination correction data thatcorresponds to a portion of the illuminated image field being captured.22. The method of claim 21, wherein combining a portion of the storedillumination correction data includes selecting the portion of thestored illumination correction data on the basis of at least a distancefrom the image field that data thereof are captured.
 23. The method ofclaim 21, wherein capturing image data of the illuminated image fieldincludes using a digital camera to do so in which the illuminationcorrection data are stored, and wherein combining a portion of thestored illumination correction data includes selecting the portion ofthe stored illumination correction data on the basis of at least adistance of the digital camera from the image field and a zoom settingof the digital camera.
 24. The method of claim 20, wherein capturingimage data of the illuminated image field includes using a digitalcamera, the illumination correction data are stored in the digitalcamera and combining the stored illumination correction data with thecaptured image data is performed within the digital camera.
 25. Themethod of claim 20, wherein capturing image data includes directing theilluminated image field onto a two-dimensional photodetector array ofpixels, and wherein retrieving stored illumination correction dataincludes referencing a table of image modification values as a functionof radial position of individual pixels from an optical reference pointand calculating radial distances of individual pixels within thetwo-dimensional photodetector array by adding a value to the radialdistance calculated for the immediately preceding scanned pixel.
 26. Themethod of claim 20, wherein combining the stored illumination correctiondata with image data includes making a calculation according to thefollowing equation:CT[x,y]=T[x,y]+IC[x,y], wherein T[x,y] is the captured image data of theilluminated image field as a function of position (x,y) across the imagefield, IC[x,y] is the stored illumination correction data as a functionof position (x,y) across the image field, and CT[x,y] is theillumination-corrected image data set as a function of position (x,y)across the image field.
 27. The method of claim 20, wherein combiningthe stored illumination correction data with image data includes makinga calculation according to the following equation:CT[x,y]=T[x,y]*IC′[x,y], wherein T[x,y] is the captured image data ofthe illuminated image field as a function of position (x,y) across theimage field, IC′[x,y] is the stored illumination correction data as afunction of position (x,y) across the image field, and CT[x,y] is theillumination-corrected image data set as a function of position (x,y)across the image field.
 28. The method of claim 20, wherein theretrieving, capturing and combining are performed within an imageacquisition device.
 29. The method of claim 28, wherein the source ofillumination across the image field is included as part of the imageacquistion device.
 30. The method of claim 29, wherein the source ofillumination includes a flash light.
 31. The method of claim 28, whereinthe illumination correction data are stored in a nonvolatile memorywithin the image acquisition device.
 32. The method of claim 28, whereinthe image acquisition device is a still camera.
 33. The method of claim28, wherein the image acquisition device is a video camera.
 34. Themethod of claim 28, wherein the image acquisition device is a personaldigital assistant (PDA).
 35. The method of claim 28, wherein the imageacquisition device is a cellular telephone.
 36. The method of claim 28,wherein the image acquisition device is handheld.
 37. An integratedcircuit chip containing circuits capable of receiving and processingdata obtained from a two-dimensional optical image detector according toa predetermined pattern, comprising: a first portion of said circuitsthat captures data from an image field of a uniform surface asilluminated by an illumination source; and a second portion of saidcircuits that generates, from the captured image data, illuminationcorrection data, to compensate for uneven distribution of light from theillumination source, as a function of position in the image field, andthat stores the illumination correction data in the imaging device. 38.An imaging device, comprising: an optical sensor having atwo-dimensional array of detectors that output data representative of anintensity of optical radiation thereon; an optical system fixed withrespect to said sensor to direct an optical radiation field onto saidsensor from an image field; an illumination source configured toilluminate the image field; one or more processors or one or morededicated image processing circuits that generate illuminationcorrection data to compensate for uneven distribution of light from theillumination source across the image field; a memory configured to storethe illumination correction data; and the one or more processors or oneor more dedicated image processing circuits further operating to modifythe data outputted from the optical sensor with the stored illuminationcorrection data, thereby to correct the optical sensor data forvariations in intensity of the light from the illumination source acrossthe image field.
 39. The imaging device of claim 38, wherein the one ormore processors or one or more dedicated image processing circuitsfurther generate the illumination correction data as a function ofposition in the image field.
 40. The imaging device of claim 38, whereinthe imaging device is a digital camera.
 41. The method of claim 38,wherein the imaging device is a still camera.
 42. The method of claim38, wherein the imaging device is a video camera.
 43. The method ofclaim 38, wherein the imaging device is a personal digital assistant(PDA).
 44. The method of claim 38, wherein the imaging device is acellular telephone.
 45. The method of claim 38, wherein the imagingdevice is handheld.